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Abstract 

This study evaluated the frequency and perceived 
effectiveness of use of three surgical simulators across all 
disciplines and years of surgical training at one institution. A 
cross-sectional survey examined specific practices and 
compared the educational experiences of 150 surgical 
residents in different training levels and disciplines of 
surgery at the University of Alberta. Resident's exposure to 
box trainers, virtual reality trainers and simulated patients 
were analyzed with both open and close ended questions. 
Eighty-eight of 150 (59%) residents participated and reported 
use of these surgical simulators varied by year and discipline 
of training. Fifty percent of respondents used box trainers in 
their surgical education, with box trainers used most within 
urology. Generally there was limited use reported with 
virtual reality trainers and simulated patients (10% of 
respondents respectively). Use of both virtual reality trainers 
and simulated patients was reportedly greatest within 
ophthalmology. Despite this limited use, residents found a 
marked and significant correlation (rs=0.78; p=0.02) between 
frequency of use and perceived effectiveness of virtual reality 
models as a method of teaching surgical skills. A marked 
degree of correlation (rs=0.65, p=0.06) between frequency of 
exposure and effectiveness was also found with simulated 
patient models with a trend towards significance. Modern 
surgical training is a complex process of cognitive and 
psychomotor skills training. Use of surgical simulators is 
expanding, and as more experience and understanding is 
gained in surgical education, enhanced teaching methods 
will ultimately produce better skilled surgeons. 
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Introduction 

Since ancient times, the axiom of "see one, do one, 
teach one" has permeated the culture of surgical 



education. A surgical trainee watches as a task is 
demonstrated, then performs the task with 
supervision prior to teaching it to someone else in 
turn. This age-old method of instruction is no longer 
accepted unquestionably As surgical techniques and 
technology continue to expand there are a greater 
number of procedures to master prior to graduation. 
New regulations also limit the number of consecutive 
hours residents are permitted to work, thus 
potentially limiting their experience. Moreover, 
patients being treated in tertiary academic centers 
tend to have more complex diseases, frequently 
rendering teaching opportunities to less experienced 
junior trainees inappropriate. Increasingly, patients 
are unwilling to be treated by novice surgeons even 
under direct supervision, as reflected by increasing 
litigation against surgeons. 14 Subsequently, 
alternatives to living patient models for surgical 
instruction are being devised and often implemented 
into surgical training programs. 

Studies abound within the literature postulating, 
testing and reaffirming various theories of skill 
acquisition, 2 ' 5 7 and educators are becoming more 
knowledgeable in the process of learning motor skills. 
More nebulous, however, are the methods of 
transmitting these skills. There has been a recent 
increase in scholarly interest in the methodologies 
best suited to instruct surgical trainees. Given the 
restrictions previously described - increasing 
litigation, diminished work hours, and an ever 
growing range of skills to master, it is not surprising 
that there is a rapid expansion of interest in this 
burgeoning area. 

Historically, surgical expertise was developed using 
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animal and cadaveric specimens as well as living 
patient models. Animal models, however, are 
becoming prohibitively expensive, difficult to access, 
and are increasingly being more scrutinized for their 
ethical use in teaching. 2 - 3 < 8 Subsequently, alternatives 
are being sought. Some alternatives under 
investigation include box trainers, virtual reality 
simulators, and patient simulators. Box trainers are 
boxes equipped with video equipment and surgical 
instruments to provide a representation of the 
abdominal, nasal, or other body cavities while 
trainees proceed through a variety of technical drills. 
Virtual reality simulators digitally enhance the box 
trainers and often provide some feedback to the 
trainee as well as providing objective assessment of 
speed, accuracy and precision of skill performance. 
Patient simulators are enhanced virtual reality models 
with representative surgical anatomy that responds to 
manipulation and provides sensory feedback to the 
trainee, the nuances of which further enhance the 
surgical experience. Aggarwal and Darzi predicted 
that, in the near future, surgeons will be able to 
perform "virtual surgery" in digital environments 
perfectly mimicking their living patients in the 
ultimate "dress rehearsal" of the procedure. 9 Future 
adaptations of this "virtual surgery" may include 
computerized or artificial intelligence that would 
intervene during the surgery on its own accord. 9 

In an era where surgical simulation is being promoted 
as a necessary innovation, 1 the question remains as to 
whether there is an extensive use of these simulators 
within surgical training programs and whether these 
are viable teaching options. To address the issue of 
surgical skill instruction, the frequency of use of three 
surgical simulators across all disciplines and years of 
surgical training at one institution as well as how 
effective surgical residents perceived these 
instructional models to be was investigated. 

Methods 

Questionnaire Development 

To answer the study objectives, a cross-sectional, 
group administered survey examined specific 
practices and compared educational experiences 
between different training levels and disciplines of 
surgery. A questionnaire with both focused 
open-ended and close-ended questions was 
developed. The qualitative responses from the 
open-ended questions permitted content analysis and 
the close-ended, Likert-styled questions quantified 



the frequency and perceived efficacy of each method 
used in surgical instruction. In order to generate a 
large sample size that could be extrapolated to 
represent surgical training programs across Canada 
and beyond, the questionnaire polled as many 
residents as possible. Initially the questionnaire was 
piloted to several staff surgeons at outside 
institutions prior to its presentation to the target 
population to ensure the questionnaire was effective 
for evaluation of the central question of surgical skills 
instruction. The final questionnaire reflected the 
intent of the study questions; it was concise, 
unambiguous and capable of eliciting information 
relevant to the purpose of the study. In order to avoid 
a biased study population, those who piloted the 
questionnaire were not part of the target population. 
This permitted the target population participants to 
remain "naive" to survey questions. 

Study Population 

To determine methods of surgical skill instruction 
employed at the University of Alberta (U of A), 
voluntary participants from the 150 surgical residents 
(spanning all years and disciplines of surgery) 
registered in all surgical disciplines were recruited. 
As the total number of surgical residents was 
relatively small, no surgical sub-specialty was 
excluded. The selected 150 surgical residents served 
as a representative sample of surgical residents 
training in Canada. Recruitment occurred on site at 
the University of Alberta's affiliated teaching 
hospitals during an academic half day for each 
department or division of surgery. Appropriate 
permission was obtained from the Faculties of 
Education, Extension and Augustana Research Ethics 
Board (EEA-REB), the Health Research Ethics Board 
(Health Panel) - University of Alberta, and the 
Residency Program Directors and the Chairs of the 
Departments of Surgery. 

Data Management 

Responses generated from the close-ended questions 
on the questionnaire were analyzed using 
non-parametric Spearman rho (rs) correlations and 
two-tailed probability tests to compare two 
categorical values (Graph Pad Prism 4, GraphPad 
Software Inc., California, United States of America). 
The degree of association was based on the five "rules 
of thumb" developed by Franzblau: 10 "r" values 
ranging from 0-0.20, 0.2-0.4, 0.4-0.6, 0.6-0.8, and 
0.8-1.0 were considered as negligible, low, moderate, 



63 



International Journal of Modern Education forum Vol. 1 Iss. 2, November 2012 



marked, and high degrees of correlation respectively. 
Open-ended questions were hand-analyzed and 
coded thematically for each teaching method. 

Results 

All 150 residents within the target population of 
surgical subspecialties had the opportunity to 
participate. Of these, a total of 88 residents completed 
and returned the survey as illustrated in Table 1. The 
overall survey response rate was 59%. Each 
instructional method was reviewed for the frequency 
of exposure to box trainers, virtual reality trainers 
and simulated patients by both surgical discipline 
and year of training. 

Box Trainer 

Fifty percent (44/88) of all responding residents 
denied ever using box trainers in surgical education, 
44% (39/88) reported using this two to three times per 
year, and 6% (5/88) used box trainer models 
monthly. Figure 1 depicts the duration of time 
residents in each program and year spent using this 
form of instruction and the level of teaching 
effectiveness is ranked by both surgical program and 
year of training. Overall, all 44 participants ranked 
the effectiveness of this method of instruction. The 
average rated effectiveness by all respondents was as 
follows: 25% (11/44) strongly agreed; 50% (22/44) 
agreed; 23% (10/44) neither 

agreed nor disagreed; and 2% (1/44) disagreed that 
box trainers were effective in teaching surgical skills. 
No correlation (rs= - 0.10) was found between 
frequency of exposure and perceived effectiveness. 
According to the surveyed responses at this 
institution, box trainers are used most frequently in 
urology as compared to other surgical subspecialties. 
Evaluating the frequency of use by year of residency 
training, 84% (16/19), 41% (7/17), 50% (11/22), 36% 
(5/14), 36% (4/11), and 25% (1/4) of participating 
residents in first, second, third, fourth, fifth and sixth 
years respectively have experience with box trainers, 
with seventh year trainee having no experience with 
this model (0/1). Twenty-seven of the 88 residents 
recorded comments about what they enjoyed most 
regarding box trainer models resulting in 29 
responses. Sixteen residents supplied 18 comments 
regarding what they liked least regarding box trainer 
models used in surgical skills instruction (Table 2a). 



TABLE 1 RESPONSE RATE BY SURGICAL SUBDIVISION AND 
YEAR OF TRAINING 
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Virtual Reality Trainer Model 

Examining all surgical subdivisions in the various 
years of training, 10% (9/88) reported experience with 
virtual reality trainers; 3% (3/88) used it two to three 
times per year; 3% (3/88) used it monthly, 2% (2/88) 
used it weekly, and only 1% (1/88) of the population 
reported more than once daily use. Of all surgical 
disciplines, ophthalmology residents reported the 
most frequent use of this teaching method. Evaluating 
the frequency of use by year of residency training, 5% 
(1/19), 2% (2/17), 10% (2/22), 7% (1/14), 27% (3/11) of 
participating residents in first, second, third, fourth, 
fifth years respectively had experience with virtual 
reality 
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TABLE 2 CHARACTERISTICS OF SURGICAL SIMULATORS 
REPORTED AS MOST AND LEAST LIKED: A) BOX TRAINER 
MODEL; B) VIRTUAL REALITY TRAINER; C) SIMULATED 
PATIENT MODEL 



a) Box Trainer 
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b) Virtual Reality Trainer 
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trainers, with the sixth and seventh year trainees 
having no experience with this model (0/4 for year six 
and 0/1 for year seven). Figure 2 depicts the time 
spent using this model and level of teaching 
effectiveness ranked by both surgical discipline and 
year of training. 

Overall, nine respondents ranked the effectiveness of 
this method of instruction. The average rated 
effectiveness for all respondents was as follows: 44% 
(4/9) strongly agreed; 22% (2/9) agreed; and 33% (3/9) 
neither agreed nor disagreed. A marked and 
significant correlation (rs=0.78; p=0.02) between 
frequency of exposure and effectiveness was noted in 
the group of residents using the virtual reality 
trainers. Six of the 88 residents recorded a total of 
seven comments on what they liked most regarding 
virtual reality training models. Seven residents 
provided seven comments regarding what they liked 
least regarding virtual reality training models used in 
surgical skills instruction (Table 2b). 

Simulated Patient Models 

In all surgical subspecialties and years of training 10% 
(9/88) of respondents reported experience with 
simulated patient models. Only 8% (7/88) reported 
using this model two to three times yearly; 1% (1/88) 
used it monthly; and only 1% (1/88) used simulated 
patients weekly. Similar to the virtual reality training 
model, ophthalmology residents used this method 
more frequently than any other surgical discipline. 
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FIGURE 1 BOX TRAINER MODEL - FREQUENCY OF BOX TRAINER USE IN TEACHING SURGICAL SKILLS BY SURGICAL DISCIPLINE (A 
AND B) AND YEAR OF TRAINING (C AND D) WITH CORRESPONDING RANK OF EFFECTIVENESS. *CV SURGERY IS CARDIOVASCULAR 
SURGERY, t ENT IS OTOLARYNGOLOGY - HEAD & NECK SURGERY, JOBS-GYNE IS OBSTETRICS & GYNECOLOGY 
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FIGURE 2 VIRTUAL REALITY TRAINER MODEL - FREQUENCY OF VIRTUAL REALITY TRAINER USE IN TEACHING SURGICAL SKILLS 
BY SURGICAL DISCIPLINE (A AND B) AND YEAR OF TRAINING (C AND D) WITH CORRESPONDING RANK OF EFFECTIVENESS. *CV 
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FIGURE 3 SIMULATED PATIENT MODEL - FREQUENCY OF SIMULATED PATIENT MODEL USE IN TEACHING SURGICAL SKILLS BY 
SURGICAL DISCIPLINE (A AND B) AND YEAR OF TRAINING (C AND D) WITH ASSOCIATED RANK OF EFFECTIVENESS. *CV SURGERY 
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Eighty-four percent (16/19) of first year residents had 
not used this method. This experience is similar to the 
remaining years with 82% (14/17) of second year 
residents, 95% (21/22) of third year residents, 86% 
(12/14) of fourth year residents, and 100% (11/11; 4/4; 
and 1/1) of fifth, sixth, and seventh year residents. 
Figure 3 depicts the amount of time spent by trainees 
and the level of teaching effectiveness ranked both by 
surgical program and year of training. Overall 9 
respondents ranked the effectiveness of simulated 
patient models. The average rated effectiveness by all 
respondents was as follows: 33% (3/9) strongly agreed; 
33% (3/9) agreed; and 33% (3/9) neither agreed nor 
disagreed with simulated patient trainer models. A 
marked degree of correlation (rs=0.65) between 
frequency of exposure and effectiveness was found 
and although there was a trend towards significance, 
this result was not statistically significant (p=0.06). Six 
of the 88 residents (6%) recorded a total of seven 
comments on what they liked most regarding full 
procedural trainers. Four residents provided five 
comments regarding what they liked least regarding 
this model used in surgical education (Table 2c). 

Discussion 

Due to ever expanding surgical techniques and 
technologies, coupled with restricted hours and 
access to complex surgical patients, along with the 
growing concerns of patient litigation, the manner in 
which surgical trainees are instructed is changing. 
Alternatives to living patients as teaching models are 
being sought and implemented in training programs 
across the nations. Among the alternatives are the 
surgical simulators and these vary from simple box 
trainers to sophisticated and costly virtual reality 
trainers. How prevalent these alternatives are within 
surgical training programs, however, has not yet been 
determined. 

Upon review of current teaching methodologies at 
one Canadian academic institution, the use of surgical 
simulators remained a relatively minor part of 
surgical education for the majority of surgical 
disciplines. Only 50% of participating residents at this 
institution (U of A) reported using box trainers 
throughout the course of their surgical training. Of all 
surgical disciplines, interestingly the urology 
residents used this method of instruction most 
frequently. In discussion with the Urology Program 
Director at the U of A, box trainers are readily 
accessible to residents at all hours as since 2000 there 
are box trainers located with resident call rooms. This 



ready access may increase the likelihood of use by the 
urology residents. Certainly use of box trainers at the 
U of A furthers the laparoscopic training in keeping 
with the work of Dechene et al. in 2006. 11 This study 
noted that, of all urologic training programs 
registered with the American Urologic Association, 
only 38% of responding residents felt their training in 
laparoscopic techniques to be "satisfactory". From 
their study, the investigators called for expanded 
laparoscopic technical training within urology 
programs. 

Only ten percent of all participating residents used 
virtual reality trainers or simulated patient models as 
part of their surgical training. Residents who used 
these high fidelity models, however, perceived them 
to be very effective in their instruction. A marked 
degree of correlation (0.78 and 0.65 with virtual 
reality trainers and simulated patients respectively) 
was found between the perceived efficacy of the 
instructional tool and the amount of time residents 
employed this tool in their training. Despite the small 
numbers of residents using this method, this 
correlation was significant (p = 0.02) for the virtual 
reality trainers and approaching significance for the 
simulated patient models (p = 0.06). It is apparent that 
the more time a resident spent with the surgical 
simulators, the more effective they felt this teaching 
method to be. 

There also appeared to be a difference in the surgical 
training model used depending on the year of 
surgical training. Interestingly, use of the simulated 
patient models was noted to be directed to the more 
junior residents. Only one of the thirty (3%) residents 
in fourth year or above used a simulated patient 
model in their education to date. The majority of 
respondents who had tried the highest fidelity model 
of instruction were in the early years of their surgical 
education. This finding likely reflects the introduction 
of the surgical simulators early in surgical residency 
as traditionally more senior trainees would spend 
more of their training time in the operating room with 
patients rather than practicing on alternate surgical 
models. 

In general, the data showed that residents found 
learning with surgical simulation a desirable situation. 
It appeared that the ability to practice on multiple 
occasions in a low pressure learning environment was 
well received by surgical trainees (Table 2). This 
finding is not overly surprising as the operating room 
as an educational forum has been previously reported 
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to be a less than ideal learning environment for junior 
trainees. 12 The high cost of mistakes to the patient, the 
rapid and often intense pace, and anxiety provoking 
environment of the operating room may not be 
conducive to effective learning. Although alternative 
models may be the solution to some of these concerns, 
these are not without their own flaws. For example, 
the greatest criticism of all surgical simulators studied 
was the lack of realism. Given the variability of 
normal and abnormal anatomy presenting in living 
patients, it is not surprising that even the currently 
available high fidelity simulators cannot recreate a 
living model. 

Implementing new training techniques into surgical 
residency programs that promote differentiated 
instruction is critical to the successful education of 
competent surgeons. A significant consideration 
prior to integrating surgical simulation models 
however, is the cost that can be associated with this. 
Purchasing these training models can vary greatly in 
cost, from relatively inexpensive box trainers to 
expensive high fidelity simulator equipment that can 
cost hundreds of thousands of dollars. To determine 
whether a new training model is indeed cost 
prohibitive, costs of training surgical residents in 
traditional manners need to be examined. Bridges and 
Diamond demonstrated in 1999 that the cost 
associated with training surgical residents in the 
operating room due to increased surgical time 
amounted to nearly $48 000 USD per surgical resident 
per year. In 1999, this amounted to $53 million USD 
per annum to train surgical residents across the 
nation. 13 Collectively, this suggests that the 
implementation of virtual reality trainers may prove 
to be more cost effective than expected. Tempered 
with the cost to the patient in the event of any errors, 
cost to health care in rehabilitation and further 
treatment, economic cost of lost income of the patient, 
and the cost of rehabilitating or retraining the health 
professionals, virtual reality training may have a 
greater benefit overall. 

Although U of A surgical trainees are comparable to 
other surgical trainees across Canada, data gathered 
in this study may not fully represent the surgical 
training models employed in other programs 
throughout the country. Additionally, there are many 
different surgical educators involved in training each 
new surgeon, and it remains unlikely that resident 
training is uniform across the country. From the study, 
however, it is evident that the beneficial use of 
different instructional methods exists within different 



training rotations, disciplines of surgery, and years of 
training at this institution. It would therefore seem 
reasonable to presume that programs from other 
institutions could vary as well to their benefit. Despite 
these variations, the general principles of surgical 
skill instruction are similar across not only regions 
but also nations and information garnered in this 
study may be useful to other institutions involved in 
the training of future surgeons. 

Conclusion 

Modern surgical training remains a complex process 
of both cognitive and psychomotor skills training. 
Use of surgical simulators in this process is 
expanding as new technologies become available and 
the drive to practice outside the surgical suite persists. 
This study examined the use and perceived 
effectiveness of three surgical simulation models 
employed in surgical resident training at a Canadian 
medical college. Our research showed that while 
surgical simulation models are just being introduced 
into the training programs, residents who have the 
opportunity to use these models perceive them to be 
effective in surgical skills transmission. As more 
experience and understanding is gained in surgical 
education, enhanced teaching methods will 
ultimately produce better skilled and competent 
surgeons, the goal of every surgical training program. 
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